1. Technical Field of the Invention
The present invention relates to a method and apparatus for coding moving pictures at a variable bit rate, and more particularly to a method and apparatus for coding moving pictures at a variable bit rate, which can control a code quantity at a variable bit rate in real time.
2. Description of the Prior Art
MPEG-2 (ISO-IEC-13818-2) has been typically known as a method of coding moving pictures with a high efficiency in the art. In this method, a picture is divided into blocks each consisting of a group of plural pixels, and each block is subject to the discrete cosine transform (DCT) to transform a signal in a spatial domain into a signal in a frequency domain. Each frequency component obtained from the discrete cosine transform, which is called a xe2x80x9cDCT coefficientxe2x80x9d, is quantized with a predetermined quantization scale, then coded in a variable length by assigning a variable length code to a quantized transform coefficient, and output as a bit stream of coded data.
FIG. 10 is a block diagram showing an arrangement of a conventional moving picture coding method for explaining an moving picture coding method on the basis of MPEG-2 (ISO-IEC-13818-2). The moving picture coding method comprises, as shown in FIG. 10, subtracter 501, discrete cosine transform (DCT) means 502, quantization means 503, variable length coder 504, bit rate controller 505, inverse quantization means 506, inverse DCT means 507, adder 508, frame memory 509, motion compensation interframe predictor 510 and selector 511.
Input images are classified into an I-picture which is encoded with intra-frame coding, a P-picture which is encoded with interframe prediction coding with only a forward prediction, and a B-picture which is encoded with bi-directional or forward and backward predictions. The input images are divided into macro blocks each consisting of 16xc3x9716 pixels and are coded on macro block basis.
First, in the I-picture, the intra-frame coding is performed without interframe prediction. At this moment, selector 511 selects a signal value xe2x80x9c0xe2x80x9d as a predicted value, and subtracter 501 outputs the same value as the signal value of the input image. An output signal value from subtracter 501 is transformed through the discrete cosine transform at DCT means 502 into a DCT coefficient, which is outputted toward quantization means 503. The DCT coefficient is quantized at quantization means 503 with a predetermined quantization scale.
The DCT coefficient quantized at quantization means 503 is output as a quantized transform coefficient to variable length coder 504, where the quantized transform coefficient is coded in a variable length. The quantized transform coefficient coded in the variable length is output as a bit stream along with other data. The quantized transform coefficient output from quantization means 503 is also fed to inverse quantization means 506 to reconstruct original image signals.
The image signals reconstructed through the inverse quantization means 506 and the inverse DCT means 507 will be employed as a reference picture on the subsequent interframe prediction, and are stored in the frame memory 509 through the adder 508. The interframe prediction is performed to P-picture and B-picture at the motion compensation interframe predictor 510.
The motion compensation interframe predictor 510 first compares the input image signals with the reference pictures stored in the frame memory 509, and performs a motion vector detection to determine a motion vector on every macro block that is obtained from dividing the input image. A prediction mode is determined per macro block on the basis of the result from the motion vector detection. In this process, it is decided whether a process of coding a macro block for P-picture and B-picture is executed preferably in the intra-mode or not.
If the prediction mode for a macro is the intra-mode, then the process of coding the macro block for P-picture and B-picture is also performed in an intra-frame coding process. On the contrary, if the prediction mode for a macro block is not the intra-mode, then the process of coding the macro block for P-picture and B-picture is perfomed by the interframe prediction using the image signals stored in the frame memory 509 as reference images. The motion compensation interframe predictor 510 performs a motion compensation interframe prediction based on the preveously determined motion vector to generate a predicted image signal corresponding to the input image signal.
The predicted image signal generated from the motion compensation interframe predictor 510 is given through the selector 511 to the adder 508 as well as the subtracter 501. The predicted image signal given to the subtracter 501 is herein described. The subtracter 501 computes a difference signal between the input image signal and the given predicted image signal. A coding process subsequently performed to the difference signal is the same as the coding process to I-picture; the difference signal is transformed at the DCT means 502 into a DCT coefficient, then is quantized at the quantization means 503.
The difference signal quantized at the quantization means 503 is fed to the variable length coder 504 to output as a bit stream and is also fed to the inverse quantization means 506. Thereafter, a reconstructing process is performed to the coded difference signal. This reconstructing process, which is performed through the inverse quantization means 506 and the inverse DCT means 507, is similar to the reconstructing process that is performed to the above described P-picture and B-picture. Namely, the reconstructed difference signal is added at the adder 508 with the predicted image signal previously given to the adder 508 to generate a reconstructed image signal.
The predicted image signal generated at the adder 508 is stored in the frame memory 509 as a reference image for use in a subsequent interframe prediction coding process. In the method represented by MPEG-2 (ISO/IEC-13818-2), the process of coding the input moving picture signal is therefore possible to reduce a redundancy in the spatial domain by means of the DCT transform as well as a redundancy in the time domain by means of the interframe prediction. Thus, moving pictures are coded efficiently. In addition, in the method represented by MPEG-2 (ISO/IEC-13818-2), the bit rate controller 505 may also be provided between the variable length coder 504 and the quantization means 503. The bit rate controller 505 acquires information of the generated code quantity from the variable length coder 504, determines a quantization scale so as to satisfy the restriction of the bit rate, and sends the quantization scale data to the quantization means 503, thereby possibly controlling the generated code quantity.
Prior art for performing the coding process of MPEG-2 (ISO/IEC-13818-2) is specifically described below. A first example of the prior art is a method of MPEG-2 Test Model (Test Model 5, ISO/ICE JTC1/SC21/WG11/N0400, April 1993), which is known as a bit rate control method in a coding system with such a quantization process as described above. This system also employs a constant bit rate coding system, wherein the code quantity is made constant per certain unit time. For example, a known system controls a code quantity using a unit of group of pictures (GOP) that is a group of a plurality of pictures to be coded. The system adjusts a quantization scale to be set per macro block that is obtained from dividing a picture into 16xc3x9716 pixels in such a manner that a code quantity generated by the coding process on each GOP becomes constant.
The first example of the prior art employs the constant bit rate coding system. Therefore, in such a scene that a larger quantity of code is to be generated, it reduces the generated code quantity by increasing the quantization scale. To the contrary, in such a scene that a smaller quantity of code is to be generated, it increases the generated code quantity by reduccing the quantization scale. The first example of the prior art, however, sets the quantization scale so as to control the code quantity per unit time to be constant. In particular, in a coding process for the scene that requires a larger quantity of code, it increases the quantization scale and reduces the generated code quantity, thereby causing a disadvantage that it degrades an image quality.
To replace the constant bit rate coding method of the above first example, a variable bit rate coding system is also proposed. In an example of the variable bit rate coding system, when a coded image is stored into a storage medium that has a limited total code quantity to be stored under the limited quantity of the total codes, the bit rate is controlled on the basis of code quantity required for maintaining or improving average image quality.
JP 6-141298A (1994) discloses a second example of the prior art, which employs the variable bit rate coding system. The second example of the prior art first sets a standard quantization scale previously, then quantizes a DCT coefficient based on the standard quantization scale, and executes a real coding process for actually coding, through a preparatory coding process for computing a code quantity to be generated. Namely, it is a xe2x80x9ctwo-pass codingxe2x80x9d system, which executes the real coding process after finishing the whole preparatory coding process, and is possible to achieve a high quality image coding because it performs re-coding after knowing the characteristic of the whole contents.
The second example of the prior art is, however, required to execute twice processes, or the preparatory and real coding processes, during coding certain video contents, which consume at least double time period, thereby causing a disadvantage that it can not perform a real time process. To the contrary, in order to execute the variable bit rate coding in real time, a xe2x80x9cone-pass codingxe2x80x9d system is also disclosed. The xe2x80x9cone-pass codingxe2x80x9d system does not employ the preparatory coding process.
A xe2x80x9cMPEG2 real time variable bit rate coding system using quantization stepsxe2x80x9d (Inada et al., the Institute of Electronics, Information and Communication Engineering (IEICE), Information and System society meeting paper, 2D 11-3 (P3), March 1998) is known as a third example of the prior art that employs the xe2x80x9cone-pass codingxe2x80x9d system in the variable bit rate coding system. The third example of the prior art performs an image coding process on the basis of MPEG-2, and sets after a certain time period a quantization scale per GOP to adjust a generated code quantity so as to be a predetermined average bit rate.
JP 10-164577A (1998) discloses a fourth example of the prior art. The fourth example of the prior art updates a code quantity that can be assigned to the subsequent several GOPs based on an actually generated code quantity. Then, it determines code quantity or quantization scale, comparing the average picture complexity up to present with the present picture complexity.
The above third and fourth examples suppress larger variations in the quantization scale, and assign the code quantity from a lower complexity scene that needs less codes to a higher complexity scene that needs much codes as a result of setting the quantization scale. Thus, the average image quality can be improved relative to the constant bit rate control system under the same average bit rate.
The above described one-pass variable bit rate coding system, however, has a disadvantage that a sequence of coding processes may possibly invite an image quality degradation. The one-pass variable bit rate coding system first computes an assignable code quantity or target rate and computes code quantity or quantization scale. Therefore, priority is given to the bit rate control within a certain time period. Accordingly, the quantization scale depending on the scene is not sufficiently assigned. As a result, the coded image quality does not become different greatly from that coded by the constant bit rate.
In addition, even in the case of an image that generates fewer codes and can be easily coded, if its code quantity is not enough for the average bit rate, the system intends to decrease the qauntization width and increase the generated code quantity. Therefore, the assigned code quantity is increased excessively even though a sufficient image quality is obtained and the codes may be consumed uselessly. As a result, a sufficient code quantity may not be assigned in such a scene that generates many codes.
Further, in the lower complexity scene, a more degraded image may possibly be detected compared to the constant bit rate coding at the same average bit rate. The reason lies in that the code quantity for low complexity scene is reduced excessively, when the coding process is limited merely to suppressing great fluctuation in the quantization scale.
An object of the present invention is to provide a variable bit rate coding method and apparatus capable of executing a variable bit rate coding in real time. It is also capable of coding with a higher image quality than that of the conventional one-pass variable bit rate coding method that also performs the variable bit rate coding in real time.
In the moving picture coding method of the present invention, input moving picture data is coded with a given quantization scale at the moving picture coding step.
Then, a complexity is computed per a first image unit from the quantization scale and the generated code quantity.
Then, an average complexity of images coded from the beginning of coding up to now is calculated on the basis of the quantization scale and the generated code quantity; selecting a smaller complexity from the complexity per the first image unit and the average complexity and computing a quantization scale of the first image unit from the average bit rate.
Then, the quantization scale is adjusted per a second image unit on the basis of the generated code quantity and an excess or shortage quantity relative to the predetermined average bit rate.
In the above-explained average complexity computing step, the average complexity may be computed on the basis of of the complexity per the first image unit.
Further, in the moving picture coding method of the present invention, the moving picture coding means codes input moving picture data with a given quantization scale and outputs the generated code quantity to the first image complexity computing means.
The first image complexity computing means computes a complexity per a first image unit on the basis of the quantization scale and the generated code quantity.
The average complexity computing means computes an average complexity of images coded from the beginning of coding up to now on the basis of the quantization scale and the generated code quantity.
The quantization scale computing means selects the smaller complexity from the complexity per the first image unit and the average complexity and computes a quantization scale of the first image unit on the basis of the predetermined average bit rate.
The quantization scale adjusting means adjusts the quantization scale per a second image unit on the basis of the quantization scale and the generated code quantity in accordance with an excess or shortage quantity relative to the predetermined average bit rate and provides it to the moving picture coding means.
In the above-explained method of the present invention, the average complexity computing means may compute an average complexity of images coded from the beginning of coding up to now on the basis of the complexity per the first image unit;
According to the present invention, it is possible to execute a variable bit rate coding in real time and achieve a high image quality coding.